Fluidic innervation sensorizes structures from a single build material

Multifunctional materials with distributed sensing and programmed mechanical properties are required for myriad emerging technologies. However, current fabrication techniques constrain these materials’ design and sensing capabilities. We address these needs with a method for sensorizing architected materials through fluidic innervation, where distributed networks of empty, air-filled channels are directly embedded within an architected material’s sparse geometry. By measuring pressure changes within these channels, we receive feedback regarding material deformation. Thus, this technique allows for three-dimensional printing of sensorized structures from a single material. With this strategy, we fabricate sensorized soft robotic actuators on the basis of handed shearing auxetics and accurately predict their kinematics from the sensors’ proprioceptive feedback using supervised learning. Our strategy for facilitating structural, sensing, and actuation capabilities through control of form alone simplifies sensorized material design for applications spanning wearables, smart structures, and robotics.


INTRODUCTION
Multifunctionality is a defining feature in the composition and forms of biological systems (1,2). For example, the xylem of vascular plants participates in water and nutrient transport while directly contributing to structural integrity and resiliency (3). Likewise, the hierarchical structure of skeletal muscles facilitates the innervation and vascularization of densely packed muscle fibers, fulfilling the actuation, proprioception, and metabolic needs of vertebrate locomotion (4,5). These materials and structures have evolved to address multiple needs in a single composite, enabling living organisms to efficiently achieve the performance required for their continued survival and evolutionary fitness. Inspired by these lessons, engineers have increasingly explored multifunctionality in materials design as a strategy to improve the performance, range of capabilities, and efficiency of a broad range of new technologies (2,6).
Motivated by these challenges, we present a strategy for fabricating multifunctional materials with programmable mechanical behaviors and distributed sensing capabilities by controlling the form of a single build material. Our strategy involves the sensorization of architected materials with open fluidic networks, which we construct via 3D printing ( Fig. 1) (25). Architected materials are a class of materials that achieve tailorable mechanical properties entirely via geometry (26). While this makes them excellent for achieving optimally programmed mechanical performance, architected materials' dependence on geometry makes them difficult to sensorize. We surmount this problem by embedding empty, air-filled fluidic networks directly into an architected material's internal structure. Once sealed, the networks' internal pressures can be measured as voltage signals during deformation and can be used as sensory feedback. Similar cavity-based fluidic sensors have been developed for tactile feedback in robotic grippers with compliant fingertips (27,28) and soft robotic tactile skins (29). However, all of these methods rely on molding-based fabrication methods, yielding relatively large sensors that only provide tactile feedback. In contrast, our sensors are cocreated with the structure during 3D printing, enabling our sensors to be incorporated within more complex geometries and for more internal measurements. Likewise, the analog electronic feedback that we receive from our sensors is distinct from other types of soft fluidic sensors that provide solely binary mechanical feedback (30,31). For example, fluidic bistable valves have enabled an electronics-free approach to reflexive tactile sensing in soft robotic grippers that autonomously grasp upon contact with an object (30) and untethered soft robots that autonomously reverse gait upon activation of the fluidic sensor with an obstruction such as a wall (31). Sensing feedback from these fluidic sensors are only compatible with fluidic logic-based controllers (30,31), while our sensors can interface with traditional electronic controllers through a pressure transducer.
Our methods present three key opportunities. First, fluidic innervation provides a straightforward route for placing, distributing, and fabricating sensors within the sparse geometries of architected materials. Second, our fluidic sensing strategy avoids the time-varying effects common to current soft sensors. Soft sensors based on conductive liquids, piezoresistive elastomers, and viscoelastic waveguides are susceptible to drift and hysteresis because of their underlying microstructures and/or physicochemical behaviors (32). Our fluidic sensing strategy sidesteps these issues by directly reading pressure changes of closed, deformable volumes patterned within the structure. Lastly, our methods enable the creation of soft robotic systems with true somatosensory capabilities by using machine learning to associate sensor feedback with deformation for proprioception. Building off our recent efforts to develop compliant materials for motorized soft robots (33)(34)(35), we use our methods to sensorize a range of structures, including a group of materials called handed shearing auxetics (HSAs) that our group has previously introduced (33). This combination of motors and fluidic sensing yields a soft robotic system with robust actuation and perception capabilities. Our system is not susceptible to failure by overpressurization or leakage as is common in fluidically actuated soft robots, allowing us to operate the device for extensive periods of time. We collect large sensorimotor datasets and develop a deep neural network to proprioceptively predict the multidegree-of-freedom (DOF) actuator's kinematics. Fluidic innervation of the HSAs' complex, sparse geometry represents an embodiment of a multifunctional construct that enables integrated structural, sensing, and actuation capabilities achieved from a single build material.

3D printing sensorized architected materials
We first demonstrate our approach by 3D printing sensorized versions of common lattice architectures (Fig. 1A) 1B, lattices with distributed fluidic networks are 3D printed from photopolymer resins using digital light processing (DLP). Nonpolymerized resin trapped within the fluidic networks during the printing process is aspirated by vacuum (movie S1), and channels are flushed with solvent and left empty. After parts are completely cured, fluidic sensor networks are connected to differential pressure transducers via elastomeric tubing (Fig. 1C) and sealed with adhesives. The fluidic channels are therefore filled with air only, and we use the perceivable pressure changes of the closed volumes as the basis of our fluidic sensors. While DLP affords the patterning resolution required to create our complex structures, the overall green body strength and printing resin's viscosity and pot life limit the overall dimensions of fluidic features that we can pattern. We investigate the fabricable sensor geometries for multiple printing resins and find that the dimensions of successfully fabricated fluidic networks are resin dependent ( fig. S1; see Materials and Methods).
According to Boyle's law, deformation of the fluidically innervated materials results in changes of the internal pressure of the networks, P, inversely proportional to volume changes (i.e., decreases via compression and increases via extension). P is measured using in-line differential pressure transducers, which report P-dependent voltages, V. We demonstrate these operating principles in Fig. 1 using the simple cubic lattice printed from an elastomeric resin (EPU 40, Carbon). The lattice has nine straight fluidic sensors running along its length (Fig. 1D). The "top" and "bottom" (Btm) sensors lie on the top and bottom row of struts in the beam, respectively. The "middle" (Mid) sensors lie in the center row of struts, at the approximate neutral plane of the beam during downward and upward bending. "Left," "middle," and "right" sensors fall in the leftmost, middle, and rightmost columns of struts. Thus, left and right sensors lie on opposite sides of the neutral plane when the beam undergoes leftward and rightward bends. As the cubic lattice undergoes manual bending (Fig. 1, E and F, and movie S2), we observe decreases in P for the sensors lying above the neutral plane (V < 0), while sensors below it report an increase in P (V > 0). Notably, middle sensors lying in the neutral plane only produce a small voltage increase, likely due to compression by the solid struts parallel to the direction of bending. In Fig. 1 (G and H) and movie S2, we explore tactile sensor response by pressing on individual columns of sensors. In all instances, we observe increases in P during compression (V > 0), which are consistent with our results from manual bending. We also observe a depth-dependent response, with sensors closer to the contact point providing greater V than those further from it. Importantly, in both sets of experiments, we observe clear, highly responsive feedback from all sensors during lattice deformation. Any instances where V does not return to V = 0 after deformation result from either the lack of precise motion in manual deformations or the intrinsic creep and stress relaxation of the proprietary viscoelastic resins (35). Over a 12-hour thermal drift study, we observed stable sensing ( fig. S3 and Supplementary Text). Figures S4 and S5 provide details of the data in Fig. 1

Mechanical characterization of sensorized lattices
We quantitatively study fluidic sensing with elastomeric, fluidically innervated body-centered cubic (BCC) and octahedral lattices undergoing compression (Fig. 2). Figure 2 (A and B) shows schematics of the BCC and octahedral lattices with five fluidic sensors of approximately equal volume (see table S1). We selected BCC (M = −13) and octahedral lattices (M = 0) for their similar architecture yet different bending-and stretching-dominated mechanical behaviors, respectively, according to Maxwell's stability criterion, M. Figure S6 and movie S3 demonstrate the individual sensitivity of the sensors, named S1 to S5, through a tactile demonstration, where sensor responses are recorded from the octahedral lattice as the lattice is manually pressed directly onto or nearby the sensors.
During compression testing (Fig. 2, C and D), we observe that the octahedral lattice is stiffer than the BCC lattice (Fig. 2, E and G). This stiffness is reflected in the higher V measured in the octahedral lattice's sensors (Fig. 2, F and H), corresponding to higher forces required for compression. In these experiments, we observe that the five sensors in each lattice behave similarly for compressive forces below approximately 100 N. Above 100 N, sensors in the octahedral lattice show greater variability with increased compression on account of the lattices' extreme, heterogeneous deformation in this regime.
To understand the dynamic response of the fluidic sensors under step compressions, we recorded sensor responses for sensorized BCC and octahedral lattices compressed to fixed distances ( Fig. 3 and fig.  S7). Figure 3 provides the dynamic sensor responses of the BCC and octahedral lattices undergoing 60-s step compressions of 20 mm (Fig. 3, Ai and Bi) and 1 mm (Fig. 3, Aiii and Biii). The corresponding mechanical responses measured are provided in Fig. 3 (A, ii and iv, and B, ii and iv). The decrease in compressive force over the 60-s hold indicates stress relaxation in the lattices. For each lattice, we observe that the overall magnitude in sensor response V decreases with decreasing compression distance. This is in agreement with Fig. 2. Even the 1-mm compressions produce a measurable V, as shown in plots (iv) of Fig. 3. We also see that the magnitude in sensor response for each compression distance is greater for the stiffer octahedral lattice than the BCC. Importantly, though, we observe a timevarying decay in V over the 60-s compression that corresponds to the stress relaxation response in Fig. 3 (A, ii and iv, and B, ii and iv). Movie S4 shows an example compression test at 20 mm with the octahedral lattice. In this video, the octahedral lattice requires several seconds to return to its initial dimensions when the compressive force is removed, revealing the extent of stress relaxation in these viscoelastic resins. Figure S7 shows additional sensor responses during step compressions at 5 and 10 mm.
Data from cyclic compression experiments in figs. S8 and S9 further reveal that the viscoelasticity of the cross-linked resins is responsible for any time-varying behavior in the sensors. Even after 10,000 cycles of compression (fig. S10), we observe largely nonhysteretic sensor responses given the fluidic sensing approach (Supplementary Text). Overall, the fluidic sensors' performance during mechanical characterization suggests that our sensorization technique is practical for architected structures and is a reliable alternative to soft matterbased conductors.

Sensorized HSAs
We next turn to the sensorization of HSAs, a recently developed class of architected materials that our group has developed for the design of motorized soft robots (33)(34)(35). Through a repeated joint linkage design, the HSA form tightly couples twisting with linear extension, enabling a single motor to drive a pair of HSAs as a compliant, soft robotic actuator. As with other architected materials, HSAs are difficult to sensorize because of their complex forms, and sensors must accommodate HSAs' extreme deformation (36). Sensorizing via fluidic innervation bypasses this issue by allowing us to embed sensors within the HSA architecture. Using a flexible polyurethane resin 4 of 10 (FPU 50, Carbon) (35), we 3D-printed two different sensorized HSAs (sHSAs), each with three embedded fluidic sensors. The first sHSA is based on a straight, unconstrained variety (Fig. 4A). The straight sHSAs' "full," "half," and "quarter" sensors weave through the sHSA structure, terminating at 1×, 0.5×, and 0.25× the sHSA length, respectively. The second sHSA is based on a bending, constrained variety (Fig. 4B). Adding constraint features in the HSA turns otherwise linear extension into out-of-plane bending (34). The bending sHSAs' sensors are called "1/4," "3/4," and "1/2." We select these asymmetric sensor designs to sense different areas and modes of sHSA deformation. All investigated sHSA design parameters are provided in table S2.
We characterized fluidic sensor responses embedded in straight sHSAs via cyclic tensile extension (Fig. 4C). sHSA extension yielded increasing V for the full and half sensors (with 1.5-mm diameters; Fig. 4D). As the sensors' diameters were increased, we observed increasing sensitivity from the full and half sensors ( fig. S11). This is expected because larger sensor volumes provide larger pressure changes during identical deformations (Supplementary Text and  table S2). Similarly, the quarter sensor's small volume leads to negligible sensitivity during linear extension in all cases. Following an analogous investigation, we found that a sensor diameter of 1 mm is appropriate for bending sHSAs given the reduced width of their widest struts (Fig. 4B). We then used two oppositely handed bending sHSAs and constructed the soft robotic finger shown in Fig. 4E,  fig. S12, and movie S5. While the sensors in this device have the smallest volume of all sensors presented in this work, we see agreement between servo input and V for the 3/4 and 1/2 sensors in Fig. 4F. The small-volume 1/4 sensors report a noisy response because of their placement giving either a low sensitivity to bending or a minimal deformation with respect to the locations of the rigid constraint features.

Learning kinematics of sHSA-based soft robots
To demonstrate a sensorized version of an electrically driven, sHSA-based soft robot, we constructed the system shown in Fig. 5A  and fig. S13. The design of this four-DOF soft robotic platform was originally introduced in (34) and is composed here of four straight sHSAs with 1.5-mm-diameter full, half, and quarter sensors. Each sHSA is actuated by a different servo motor, with neighboring sHSAs having opposite handedness. Thus, the overall soft robot has 12 fluidic sensors and four servos for actuation. Figure 5 (A and B) 0 mm 1 0 mm 2 0 mm 0 mm 1 0 mm 2 0 mm Loading Unloading S1 S2 S3 S4 S5 Loading Unloading S1 S2 S3 S4 S5 S1 S2 S3 S4 S5 S5 S4 S3 S2 S1 S1 S2 S3 S4 S5 S5 S4 S3 S2 S1  Despite an increasing focus on data-driven approaches for solving longstanding challenges in soft robot control (21,23,24,37,38), the robustness and reliability of current soft robotic actuation and sensing strategies limit the size of datasets that can be acquired for proprioceptively determining shape or kinematics in soft sensorized robots. For example, popular pneumatically actuated soft robots are notoriously susceptible to premature failure from overpressurization with repeated actuation, making it challenging to build sufficiently large datasets. To evaluate the robustness of our system in alleviating performance challenges in data-driven approaches to soft robotic perception, we drove our soft robot through varying extents of its four DOFs for over 18 hours of sensorimotor experimentation, measuring sensor responses against a motion capture ground truth. This long data collection period is notably large for a soft robotic system. As described in the Materials and Methods, we then used this large, rich dataset to train a long short-term memory (LSTM)-based deep neural network capable of proprioceptively predicting the soft robot's kinematics solely based on the data from 12 fluidic sensors as model inputs ( fig. S16). Other data-driven sensing pipelines based on soft sensors with time-varying, hysteretic behaviors require neural networks with large numbers of hidden layers (21,23). We achieve accurate pose predictions of our sHSA platform with a simple network architecture. Figure 5 (C and D) and movie S7 show animated representations of our ground truth and model-predicted rigid body pose of the platform with corresponding Euclidean distance (i.e., position) error and rotational angle error over the course of representative time series (see fig. S18 for error explanation). While we certainly observe outlying instances, we generally observe that kinematic predictions align well with ground truth when soft robot motion is more continuous and holding of a pose is minimized. We suspect that this is due to the use of voltage changes as model inputs, as opposed to absolute voltage values. For interpreting the prediction data shown in Fig. 5 (C and D) and movie S7, we note that the rest length of the sHSA platform is 120 mm, and the maximum vertical extension is approximately 40 mm. The overall position error in Fig. 5D is small compared to these lengths. Thus, our motorized sHSAs and fluidic sensing strategies allow us to bring robust actuation and perception capabilities to soft robotics, bypassing both of these problematic issues for the field.

DISCUSSION
We have demonstrated the versatility and integrity of our fluidic sensing strategy across a wide range of passive and active architected materials. Using a DLP-based printing method, we simultaneously create sensorized, architected materials and innervating vascular networks for fluidic sensing, bypassing the need for copatterning structural materials with conductive materials that have extreme time-varying behaviors. Importantly, following a bioinspired approach to multifunctional material design, our strategy of sensorizing structures through fluidic innervation greatly simplifies how materials with self-sensing and programmed mechanical behaviors can be fabricated. Through the sensorization of HSAs, we also introduce a soft robotic system representing the first demonstration of how a single build material can be architected to simultaneously achieve tailored structural properties, distributed sensing, and actuation capabilities. While many future sensorized materials and structures may require multiple materials for their design specifications, our method showcases a single-material approach to sensorizing architected materials, an important class of materials whose sparse, complex geometries complicate conventional approaches for fabricating sensing networks throughout. Moreover, many of these materials are already predominantly fabricated with DLP and other light-based printing methods, making our approach a strong fit for empowering future material research.
Coupling the simplicity of our sensorization approach with other digitally fabricated materials could herald fundamentally new opportunities for the design and manufacturing of sensorized materials for wearable devices, smart structures, autonomous bioinspired robots, and beyond. . Pressurized air is used to verify successful leak-proof adhesion of the tubing to the sHSA, and sensor outlets are closed by filling with a small amount of Sil-Poxy, creating a closed volume. Sensor tubing is then joined to differential pressure sensors by threading the sensor tubing into larger diameter tubing placed over the sensors' ports. Three differential pressure sensors (Amphenol, All Sensors) are used to measure pressure changes in the ranges of 0 to 0. . Range 0 to 5 in H 2 O differential pressure sensors is used for the fluidically innervated cubic, BCC, and octahedral lattices. Range 0 to 1 in H 2 O pressure sensors is used for the full and half sensors of straight sHSAs, as well as the 3/4 and 1/2 sensors of the bending sHSAs. Range 0 to 0.5 in H 2 O pressure sensors is used for the quarter and 1/4 sensors of the straight and bending sHSAs, respectively. All output voltage signals from differential pressure sensors are filtered by a low-pass filter (cutoff frequency of ≈30 Hz) and read by a digital acquisition unit (NI USB-6212 DAQ, National Instruments). All reported sensor outputs are unfiltered.

3D printing fluidically innervated architected materials
The differential pressure sensors have two ports. The nominal voltage at a zero differential pressure (i.e., when both ports are at atmospheric pressure) is 2.25 V (specification sheet, Miniature Amplified Low-Pressure Sensors, All Sensors). Therefore, one can select which port is connected to the fluidic sensor to achieve an increasing/decreasing voltage change with respect to 2.25 V for an increasing pressure in the fluidic sensor. As discussed in greater detail in Supplementary Text, we mitigate effects due to thermal drift by connecting the second port of the differential pressure sensor to a dummy line, which comprises identical silicone tubing that wraps around the tubing to the functional fluidic sensor. After all tubing are sealed and connected to the differential pressure sensor, we allow the sensors to reach equilibrium internal pressures for 48 hours, which is the time point on which a differential pressure sensor connected to a fluidic network reports a nominal voltage of approximately 2.25 V.

Print parameter characterization
The pressure drop, P, required to remove all remaining resin via vacuum is dependent on the length, L, and radius, R, of the network, per the Hagen-Poiseuille equation BCC and octahedral lattices undergo two rounds of break-in compression cycles before further characterization: 100 cycles of compression to 10 mm at 1 mm/s and 100 cycles of compression to 20 mm at 5 mm/s. The lattices are then compressed at 2.5-mm increments from 0 to 20 mm and then decompressed from 20 to 0 mm. At each (de)compression interval, a force measurement is recorded from the Instron, and the sensor readings are recorded for 10 s, over which a mean and SD output voltage is determined for each sensor. This process is repeated three times.
Step compressions on BCC and octahedral lattices are conducted on an Instron. The lattices are compressed at 20 mm/s by 20, 10, 5, and 1 mm for 60 s. These compression distances correspond to compressive strains of 0.4, 0.2, 0.1, and 0.02, respectively, as the lattice structures are approximately 50 mm in height. Compressive load is removed at 20 mm/s. Sensor readings are taken before, during, and after the step compression.
Lastly, the BCC lattice was cyclically compressed 10,000 cycles to 10 mm at 1 mm/s. For this study, sensor readings were recorded at 5 Hz.

Characterization of sHSAs
Sensor responses from straight sHSAs are characterized using an Instron during cyclic, uniaxial extension. Straight sHSAs with fluidic sensor diameters of 0.75, 1, 1.25, 1.5, and 1.75 mm were studied. Custom adapters are used to mount the straight sHSAs to the Instron to allow for free rotation of one end of the sHSA while fixing the other. Straight sHSAs are stretched in 5-mm increments from 0 to 50 mm and then back to 0 mm, with sensor readings recorded from each sensor by measuring the mean and SD of the reading over 10 s of sampling at 50 Hz. Experiments are repeated three times. Before all experiments, straight sHSAs are cyclically extended 50 mm at 10 mm/s for 100 cycles as break-in to remove any strain-softening behaviors due to the Mullins effect. sHSAs are then cycled at 1 mm/s for 10 cycles to validate that no leaks are present.
Bending sHSA assemblies are actuated using Dynamixel MX-28 servos (ROBOTIS) and are controlled via MATLAB (MathWorks). Two sHSAs, each of opposite handedness, are counterrotated against each other in a geared setup with a single servo. Sensor readings are collected at ≈15 Hz.

sHSA soft robotic platform
Soft robotic fingers and multi-DOF platforms are constructed [as previously described in (35)] from sHSAs. Briefly, sHSA fingers and platforms are based on 2 × 1 and 2 × 2 assemblies of sHSAs with opposite handedness, respectively. Fingers are driven by one servo motor, while platforms are driven by four servo motors (i.e., one servo per sHSA). All soft robotic systems are actuated using Dynamixel MX-28 servos (ROBOTIS) and are controlled via MATLAB (MathWorks).

Data collection with sHSA platform
Data for neural network prediction of the sHSA platform's pose were taken by driving the platform through a series of motions while simultaneously recording sensor values against a motion capture ground truth. For all experiments, the sHSA platform is actuated through a sequence of motions, returning to its neutral position after each one. The motions are elongation, contraction, bend forward (or north bend), bend backward (or south bend), bend left (or west bend), bend right (or east bend), clockwise twist, and counterclockwise twist. For each trial, all of the motions are performed, but the order in which they are performed is randomized.
During these experiments, output voltage signals from differential pressure sensors are recorded by a digital acquisition unit (NI USB-6212 DAQ, National Instruments) through MATLAB (MathWorks). MATLAB also directly records servo position feedback from Dynamixel MX-28 servos (ROBOTIS). Ground truth readings are recorded through rigid body motion tracking via Motive (OptiTrack). Data across software are synchronized via the interpolation of UNIX time stamps associated with each measurement, resulting in a final sampling frequency of 15 Hz. To normalize across trials, the initial sensor measurement for each trial is recorded as 0 V, so further sensor readings are reported as the pressure difference in the fluidic sensors.
To add variance across trials, a specific servo velocity, extent of motion range, and end-of-movement hold time are chosen from a preselected list of options. This causes each trial to be of different lengths, making it harder for the neural network to track spurious patterns. For example, a faster servo velocity will result in the sequence being completed in less time. The servo speeds studied were 5, 10, 20, and 40 rpm. The hold times studied were 0, 5, and 10 s. The range fractions studied were 25, 50, 75, and 100% of full DOF range. For each given velocity, range, and hold time, five trials were conducted.
In summary, 240 trials were autonomously recorded over a single 7-hour period. Nineteen of these trials did not have accurate synchronization between the motion capture and MATLAB recordings and were omitted. Nine more of these trials were found to have errors in the data concatenation process and were omitted as well. These experiments formed the dataset used for the neural network training, testing, and validation. A total of 11 hours of additional experiments was conducted to debug the data collection script, to break in and calibrate the sHSAs, and to take videos and photographs.

Learning problem
To estimate the forward kinematics of the four-DOF sHSA platform, a neural network that predicts its pose (i.e., position and orientation) solely using analog voltage readings from the fluidic sensors as input is developed. Because time-dependent effects such as stress relaxation and creep are inherent to viscoelastic materials used for HSAs, the input-output relation is modeled with LSTM networks, a class of neural networks commonly used for learning-based proprioception in soft robotics because of their ability to capture temporal relations (21,23,24,37).

Data preprocessing
Of the 212 remaining time sequences, the last 32 are set aside as the test set. Because these test data are neither used for model training nor for model selection, evaluating the final LSTM on it provides an unbiased estimate of the system performance in deployment. The data from the remaining 180 experiments account for ≈85% of the total data and form the basis for the training and validation sets. Because the longest of these recordings consists of 3688 time steps, which is difficult to learn even for LSTMs, sequences of a more manageable length are extracted by sliding a window of length 64 over each recording ( fig. S15). To simultaneously augment the dataset, the window is slid with a stride of 20. The first data point in each recording (zero voltage difference for input data; initial pose for output data) is duplicated until the length of the recording equals 64 plus a multiple of 20. In this way, a total of 14,528 sequences with length 64 are generated. A total of 1453 (≈10%) are split off randomly as a validation set to evaluate a particular choice of hyperparameters during the model selection process. Lastly, training and validation sets are shuffled, inputs are normalized to zero mean and unit variance, and outputs are normalized to zero mean.

Neural network architecture
The neural network reads in a time sequence of 12D vectors (sensor voltages) and outputs a time sequence of 7D vectors (platform pose) with equal length (fig. S16). The inputs are first passed through stacked LSTM layers, where the number of layers and the number of hidden and cell states are tunable hyperparameters. The output of the LSTM layers are then passed through a fully connected layer with equal input and output dimensions, followed by ReLU (rectified linear unit) activation. The data are passed through another fully connected layer that outputs a 7D vector. Lastly, a normalization layer is applied to the four values corresponding to the orientation prediction because an orientation is given by a 4D vector that represents the coefficients of a unit quaternion. To mitigate overfitting, the first fully connected layer and all LSTM layers are equipped with a dropout probability of 0.2.
Because the coefficients of unit quaternions can be interpreted as points on a 4D hypersphere, the distance between two quaternions are described by the geodesic distance on the hypersphere. Furthermore, by definition of quaternions, antipodal points on the hypersphere describe the same orientation in 3D Euclidean space. However, the simple mean squared error (MSE) is found to be sufficient in practice, although it neglects both the curvature of the hypersphere and the symmetry of quaternions. This choice is reasonable as long as training converges, because the Euclidean distance approximates the geodesic distance for quaternions close to each other. The loss between predicted and ground truth pose is therefore computed as the sum of the position MSE and the quaternion MSE, although additionally weighted by a hyperparameter that balances the relative importance of minimizing these errors.

Model selection
Starting around an initial configuration found by following the best practice guidelines for training LSTMs (39), a grid search over the following hyperparameters is performed: quaternion weight (10, 100, and 1000), initial learning rate (0.001, 0.005, and 0.01), number of LSTM layers (1, 2, and 3), and number of hidden and cell states (50, 100, 150, and 200). Implemented in PyTorch, all models are trained with the Adam optimizer and a batch size of 32 for 100 epochs. The learning rate scheduler halves the learning rate when the validation error does not improve for 20 epochs. To avoid overfitting, an early stopping rule observing the loss on the validation set is used. In other words, while training with a particular choice of hyperparameters, only the model that shows the lowest validation error across all 100 epochs is retained. To account for randomness during neural network training, the validation loss for a particular choice of hyperparameters is averaged over a total of three runs. Lastly, the model with the lowest validation error across all possible choices of hyperparameters is retrained for 1000 epochs. Out of three runs, the run with the lowest validation loss is kept as the best model for evaluation on the test set. The summary statistics of this hyperparameter search can be found in table S1 and fig. S17.